Data processing of input/output (“I/O”) intensive applications, including enterprise applications such as data warehousing, On-Line Transaction Processing Online Transaction Processing (“OLTP”) and Relational Data Base Management Systems (“RDBMS”), are the mainstay of capital intensive in-house Information Technology (“IT”) infrastructures. High-performance data storage is a critical component of such applications.
With regard to data storage, the term “performance” means how quickly the storage system can store and/or retrieve data. This measurement is particularly important for I/O intensive applications, because such applications typically perform thousands or millions of I/O operations (writes to storage or reads from storage) per second. Thus, in such contexts, storage performance typically is measured in I/O operations per second (“IOPS”).
Unfortunately, data storage technology has not advanced at the same pace as compute technology, creating a performance gap that results in unbalanced systems and constrains overall performance. Typically, IT departments attempt to address this imbalance by adding expensive storage capacity to shrink the I/O performance gap. In traditional storage sub-systems, an increase in I/O performance is achieved by adding disk (i.e. spindles) and writing sparsely across all allocated disks. This results in underutilized capacity to achieve performance.
Cloud service providers currently offer storage solely by subscribed capacity that is shared amongst many tenants, essentially also sharing and reducing performance proportionally. This practice, known as “thin provisioning,” renders many cloud storage solutions unsuitable for I/O intensive data processing. As a result, customers are left with only two options—custom-built and maintained high-performance storage solutions, which can be prohibitively expensive, and shared “cloud” storage, which generally cannot provide acceptable performance for I/O intensive applications. For example, current cloud storage offerings, while relatively inexpensive, generally provide performance around 1000-3000 IOPS, which is unacceptable for many applications. Conversely, custom-built storage solutions can provide performance in excess of 1,000,000 IOPS, but such solutions run into the millions of dollars and therefore are prohibitively expensive for many enterprises.
Thus, there is a need for more robust and flexible storage solutions that can provide a balance between capacity, performance, and cost, to allow customers to select an appropriate balance for their specific applications.